Groestl Distinguishing Attack: A New Rebound Attack of an AES-like Permutation

We consider highly structured truncated differential paths to mount a new rebound attack on Grøstl-512, a hash functions based on two AES-like permutations, P1024 and Q1024, with non-square input and output registers. We explain how such differential paths can be computed using a Mixed-Integer Linear Programming approach. Together with a SuperSBox description, this allows us to build a rebound attack with a 6-round inbound phase whereas classical rebound attacks have 4-round inbound phases. This yields the first distinguishing attack on a 11-round version of P1024 and Q1024 with about 272 computations and a memory complexity of about 256 bytes, to be compared with the 296 computations required by the corresponding generic attack. Previous best results on this permutation reached 10 rounds with a computational complexity of about 2392 operations, to be compared with the 2448 computations required by the corresponding generic attack.